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To say that a reinforcement is contingent upon a response 
may mean nothing more than that it follows the response. It 
may follow because of some mechanical connection or be- 
cause of the mediation of another organism; but conditioning 
takes place presumably because of the temporal relation only, 
expressed in terms of the order and proximity of response and 
reinforcement. Whenever we present a state of affairs which 
is known to be reinforcing at a given drive, we must suppose 
that conditioning takes place, even though we have paid no 
attention to the behavior of the organism in making the 
presentation. A simple experiment demonstrates this to be 
the case. 

A pigeon is brought to a stable state of hunger by reducing it to 75 
percent of its weight when well fed. It is put into an experimental 
cage for a few minutes each day. A food hopper attached to the cage 
may be swung into place so that the pigeon can eat from it. A solenoid 
and a timing relay hold the hopper in place for five sec. at each 
reinforcement. 

If a clock is now arranged to present the food hopper at regular 
intervals with no reference whatsoever to the bird’s behavior, operant 
conditioning usually takes place. In six out of eight cases the resulting 
responses were so clearly defined that two observers could agree 
perfectly in counting instances. One bird was conditioned to turn 
counter-clockwise about the cage, making two or three turns between 
reinforcements. Another repeatedly thrust its head into one of the 
upper corners of the cage. A third developed a ‘tossing’ response, as 
if placing its head beneath an invisible bar and lifting it repeatedly. 
Two birds developed a pendulum motion of the head and body, in 
which the head was extended forward and swung from right to left 
with a sharp movement followed by a somewhat slower return. The 
body generally followed the movement and a few steps might be 
taken when it was extensive. Another bird was conditioned to make 
incomplete pecking or brushing movements directed toward but not 
touching the floor. None of these responses appeared in any noticeable 
strength during adaptation to the cage or until the food hopper was 
periodically presented. In the remaining two cases, conditioned re- 
sponses were not clearly marked. 

The conditioning process is usually obvious. The bird happens to 
be executing some response as the hopper appears; as a result it tends 
to repeat this response. If the interval before the next presentation is 
not so great that extinction takes place, a second ‘contingency’ is 
probable. This strengthens the response still further and subsequent 
reinforcement becomes more probable. It is true that some responses 
go unreinforced and some reinforcements appear when the response 
has not just been made, but the net result is the development of a 
considerable state of strength. 


Editor's Note. This article is a reprint of an original work pub- 
lished in 1948 in the Journal of Experimental Psychology, 38, 168— 
172. 


With the exception of the counter-clockwise turn, each response 
was almost always repeated in the same part of the cage, and it 
generally involved an orientation toward some feature of the cage. 
The effect of the reinforcement was to condition the bird to respond 
to some aspect of the environment rather than merely to execute a 
series of movements. All responses came to be repeated rapidly 
between reinforcements — typically five or six times in 1 5 sec. 

The effect appears to depend upon the rate of reinforcement. In 
general, we would expect that the shorter the intervening interval, the 
speedier and more marked the conditioning. One reason is that the 
pigeon’s behavior becomes more diverse as time passes after reinforce- 
ment. A hundred photographs, each taken two sec. after withdrawal 
of the hopper, would show fairly uniform behavior. The bird would 
be in the same part of the cage, near the hopper, and probably 
oriented toward the wall where the hopper has disappeared or turning 
to one side or the other. A hundred photographs taken after 10 sec., 
on the other hand, would find the bird in various parts of the cage 
responding to many different aspects of the environment. The sooner 
a second reinforcement appears, therefore, the more likely it is that 
the second reinforced response will be similar to the first, and also 
that they will both have one of a few standard forms. In the limiting 
case of a very brief interval the behavior to be expected would be 
holding the head toward the opening through which the magazine 
has disappeared. 

Another reason for the greater effectiveness of short intervals is 
that the longer the interval, the greater the number of intervening 
responses emitted without reinforcement. The resulting extinction 
cancels the effect of an occasional reinforcement. 

According to this interpretation the effective interval will depend 
upon the rate of conditioning and the rate of extinction, and will 
therefore vary with the drive and also presumably between species. 
Fifteen sec. is a very effective interval at the drive level indicated 
above. One min. is much less so. When a response has once been set 
up, however, the interval can be lengthened. In one case it was 
extended to two min., and a high rate of responding was maintained 
with no sign of weakening. In another case, many hours of responding 
were observed with an interval of one min. between reinforcements. 

In the latter case, the response showed a noticeable drift in topog- 
raphy. It began as a sharp movement of the head from the middle 
position to the left. This movement became more energetic, and 
eventually the whole body of the bird turned in the same direction, 
and a step or two would be taken. After many hours, the stepping 
response became the predominant feature. The bird made a well 
defined hopping step from the right to the left foot, meanwhile turning 
its head and body to the left as before. 

When the stepping response became strong, it was possible to 
obtain a mechanical record by putting the bird on a large tambour 
directly connected with a small tambour which made a delicate 
electric contact each time stepping took place. By watching the bird 
and listening to the sound of the recorder it was possible to confirm 
the fact that a fairly authentic record was being made. It was possible 
for the bird to hear the recorder at each step, but this was, of course, 
in no way correlated with feeding. The record obtained when the 
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magazine was presented once every min. resembles in every respect 
the characteristic curve for the pigeon under periodic reinforcement 
of a standard selected response. A well marked temporal discrimina- 
tion develops. The bird does not respond immediately after eating, 
but when 10 or 15 or even 20 sec. have elapsed it begins to respond 
rapidly and continues until the reinforcement is received. 

In this case it was possible to record the ‘extinction’ of the response 
when the clock was turned off and the magazine was no longer 
presented at any time. The bird continued to respond with its char- 
acteristic side to side hop. More than 10,000 responses were recorded 
before ‘extinction’ had reached the point at which few if any responses 
were made during a 10 or 15 min. interval. When the clock was again 
started, the periodic presentation of the magazine (still without any 
connection whatsoever with the bird’s behavior) brought out a typical 
curve for reconditioning after periodic reinforcement, shown in Fig. 
1). The record had been essentially horizontal for 20 min. prior to 
the beginning of this curve. The first reinforcement had some slight 
effect and the second a greater effect. There is a smooth positive 
acceleration in rate as the bird returns to the rate of responding which 
prevailed when it was reinforced every min. 

When the response was again extinguished and the periodic pre- 
sentation of food then resumed, a different response was picked up. 
This consisted of a progressive walking response in which the bird 
moved about the cage. The response of hopping from side to side 
never reappeared and could not, of course, be obtained deliberately 
without making the reinforcement contingent upon the behavior. 

The experiment might be said to demonstrate a sort of 
superstition. The bird behaves as if there was a causal relation 
between its behavior and the presentation of food, although 



TIME 


Figure l . ‘Reconditioning’ of a superstitious response after extinc- 
tion. (The response of hopping from right to left had been thoroughly 
extinguished just before the record was taken. The arrows indicate 
the automatic presentation of food at one-min. intervals without 
reference to the pigeon’s behavior.) 


such a relation is lacking. There are many analogies in human 
behavior. Rituals for changing one’s luck at cards are good 
examples. A few accidental connections between a ritual and 
favorable consequences suffice to set up and maintain the 
behavior in spite of many unreinforced instances. The bowler 
who has released a ball down the alley but continues to behave 
as if he were controlling it by twisting and turning his arm 
and shoulder is another case in point. These behaviors have, 
of course, no real effect upon one’s luck or upon a ball half 
way down an alley, just as in the present case the food would 
appear as often if the pigeon did nothing — or, more strictly 
speaking, did something else. 

It is perhaps not quite correct to say that conditioned 
behavior has been set up without any previously determined 
contingency whatsoever. We have appealed to a uniform 
sequence of responses in the behavior of the pigeon to obtain 
an over-all net contingency. When we arrange a clock to 
present food every 15 sec., we are in effect basing our rein- 
forcement upon a limited set of responses, which frequently 
occur 15 sec. after reinforcement. When a response has been 
strengthened (and this may result from one reinforcement), 
the setting of the clock implies an even more restricted con- 
tingency. Something of the same sort is true of the bowler. It 
is not quite correct to say that there is no connection between 
his twisting and turning and the course taken by the ball at 
the far end of the alley. The connection was established before 
the ball left the bowler’s hand, but since both the path of the 
ball and the behavior of the bowler are determined, some 
relation survives. The subsequent behavior of the bowler may 
have no effect upon the ball, but the behavior of the ball has 
an effect upon the bowler. The contingency, though not 
perfect, is enough to maintain the behavior in strength. The 
particular form of the behavior adopted by the bowler is due 
to induction from responses in which there is actual contact 
with the ball. It is clearly a movement appropriate to changing 
the ball’s direction. But this does not invalidate the compari- 
son, since we are not concerned with what response is selected 
but with why it persists in strength. In rituals for changing 
luck the inductive strengthening of a particular form of be- 
havior is generally absent. The behavior of the pigeon in this 
experiment is of the latter sort, as the variety of responses 
obtained from different pigeons indicates. Whether there is 
any unconditioned behavior in the pigeon appropriate to a 
given effect upon the environment is under investigation. 

The results throws some light on incidental behavior ob- 
served in experiments in which a discriminative stimulus is 
frequently presented. Such a stimulus has reinforcing value 
and can set up superstitious behavior. A pigeon will often 
develop some response such as turning, twisting, pecking near 
the locus of the discriminative stimulus, flapping its wings, 
etc. In much of the work to date in this field the interval 
between presentations of the discriminative stimulus has been 
one min. and many of these superstitious responses are short- 
lived. Their appearance as the result of accidental correlations 
with the presentation of the stimulus is unmistakable. 
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